Method and apparatus for automatic white balance

ABSTRACT

A method of automatic white balance for an image capture system is disclosed. The automatic white balance mechanism ascertains the illuminant source of an image by analyzing the number of white pixels within a predefined white area in a color space diagram. The automatic white balance mechanism also determines gain adjustments based on the evaluating the average RGB values to achieve white balance.

TECHNICAL FIELD

The present invention relates to an automatic white balance mechanism inan image capture system. The system employs an automatic white balancealgorithm that determines the illuminant source of the image and adjuststhe gain on each color channel to obtain equivalent red (R), green (G),and blue (B) values. When the RGB values are equivalent and whitebalance is achieved, a white object in an image will appear white evenunder different illuminant sources.

BACKGROUND

The human visual system adapts to changing illuminant sources byensuring that a white object appears white. When a white object travelsfrom daylight, which has more blue color component, to incandescentlight, which has more red color component, the human visual system makesadjustment to balance the red, green, and blue color components toensure that a white object appears white in both daylight andincandescent light. The technique of balancing the red color, greencolor, and blue color components is known as white balance. Thus, thehuman visual system automatically white balances an image to preservethe true white color of a white object in the image as the objecttravels under different illuminant types. Image capture systems useautomatic white balance algorithms to mimic the human visual mechanismin order to reproduce the true white color of a white object in an imageunder different illuminant sources.

The strength of the RGB color components varies significantly indifferent light conditions. There is far more blue color component indaylight than in interior cool white fluorescent (CWF) light. Table 1provides a color temperature index for different illuminant types.Higher color temperature, such as daylight, has more blue colorcomponent while lower color temperature, such as incandescent light, hasmore red color component. TABLE 1 Color Temperature Index IlluminantType Color Temperature Daylight 5000-7500 K Cool White Fluorescent 4500K U30 (General Office Light) 3000 K A (Incandescent Light) 2000 K

Prior art automatic white balance methods assume that the whole imageneeds to be white balanced. This assumption causes the over inclusion ofRGB values of all pixels of an image in calculating the average RGBvalues. The average RGB values are used to adjust color gains in acaptured image. In other words, the amount of color gain to apply toeach color channel is based on making the red, green, and blue colorcomponents equal to the average RGB values. When RGB values of allpixels are included in calculating the average RGB values, undesirableinfluence from strong colors will also be incorporated. When a strongcolor object enters or leaves a scene, its influence will skew theaverage RGB values. A strong color contribution in the average RGBcalculation can ultimately cause an object to lose its true color. Forexample, when a red object enters a scene with a red background, thisimage will have a predominant red color value. The red color willheavily influence the average RGB values of this image. The red colorcontribution in the average RGB values is so strong that the affect onthe gain adjustment can cause an object to lose its true color.

Another prior art method of automatic white balancing defines a singlewhite area in a color space diagram for all illuminant types in anattempt to combat strong color influences. This method uses a colorspace diagram to identify the white pixels of an image. The white areain a color space diagram serves as a template for detecting the whitepixels of an image. If a pixel has a value falling within the white areathen it is determined to be a white pixel and its RGB values will beused to calculate the average RGB values for color gain adjustments.

A drawback of using a color space diagram with a single white area isthe possibility of incorrectly including non-white pixels in calculatingthe average RGB value. In some instance, a strong color pixel hassimilar attribute as a white pixel and can fall within the white area ofa color space diagram. The non-white pixels can have a negative effecton the RGB averaging calculation. For example, strong blue pixels havesimilar characteristics as white pixels in daylight. When an imagecontains strong blue pixels, they will incorrectly be construed as whitepixels and their RGB values will be included in the RGB averagingcalculation. The contribution from the strong blue pixels will result inincorrect average RGB values, which will be used for determining gainadjustments.

Additionally, this method cannot be used to ascertain the illuminantsource of the image because a single white area for all illuminant typedoes not have sufficient information to support further analysis toobtain the identity of the illuminant source.

Thus, there is a need for a robust automatic white balance mechanismthat can eliminate strong color influences and has the capability torespond quickly to changes in illuminant source.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and many of the attendant advantages of thisinvention will become more readily appreciated as the same becomesbetter understood by reference to the following detailed description,when taken in conjunction with the accompanying drawings, wherein:

FIG. 1 is a schematic diagram of an image capture system in accordancewith the present invention.

FIG. 2 is a schematic diagram of a color processing system withautomatic white balance.

FIG. 3 is an illustration of a color space diagram with differentpredefined white areas. This color space diagram contains predefinedwhite areas for daylight, CWF, and A/U30 illuminant types in accordancewith the present invention.

FIG. 4 is a color chart for defining white areas in a color spacediagram in accordance with the present invention.

FIG. 5 is a flow diagram of a method for predefining white areas in acolor space diagram in accordance with the present invention.

DETAILED DESCRIPTION

In the detailed description provided below, numerous specific detailsare provided to provide a thorough understanding of embodiments of theinvention. One skilled in the relevant art will recognize, however, thatthe invention can be practiced without one or more of the specificdetails, or with other methods, components, materials, etc. In otherinstances, well-known structures, materials, or operations are not shownor described in detail to avoid obscuring aspects of the invention.

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the present invention. Thus, theappearances of the phrases “in one embodiment” or “in an embodiment” invarious places throughout this specification are not necessarily allreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics may be combined in any suitable manner inone or more embodiments.

The automatic white balance (AWB) methodology analyzes a captured imageto determine its illuminant source and then, derives the amount of gainadjustment necessary to achieve white balance. The AWB mechanismexamines the pixels of an image to obtain information on the illuminantsource of the image. It also determines the gain adjustments needed forwhite balancing the image. The white pixels of an image containinformation used to ascertain the white balance setting.

The AWB mechanism uses a color space diagram that contains two or morepredefined white areas for different illuminant sources. A predefinedwhite area characterizes the space occupied by the white pixels of anilluminant source in a color space diagram. In other words, thepredefined white area is the area in a color space diagram where allwhite pixels of a particular illuminant source can be found. Therefore,locating the predefined white area of a white pixel will reveal theilluminant source associated with the pixel. A color space diagram withseparate predefined white areas for different illuminant sources canserve as a template for ascertaining the illuminant source of whitepixels.

The AWB mechanism also relies on the color space diagram to select thewhite pixels of an image. The RGB values of white pixels are used todecide whether there is a need for gain adjustments to the red, green,and blue channels to white balance an image.

AWB calculates the Green/Red (G/R) ratio and Green/Blue (G/B) ratio of apixel. AWB plots the G/R and G/B ratios on a color space diagram. Theratios will provide a point in the color space diagram. If the pixel isa white pixel, the point created by the G/R and G/B ratios will fallwithin one of the predefined white area. This analysis identifieswhether the pixel is a white pixel and identifies the predefined whitearea associated with the white pixel, which in turn provides illuminantsource information. An AWB system can implement this analysis by storingthe G/R and G/B ratios of each point in a predefined white area in atable or matrix.

When AWB identifies a white pixel of an image, the RGB values of thepixel are stored until all the pixels have been analyzed. After all theRGB values are collected, the AWB mechanism calculates average RGBvalues for all the white pixels. For efficiency, a selected group ofwhite pixels, such as every third or fourth white pixel, is used for theaveraging calculations. Average R value, average G value and average Bvalue are calculated for the white pixels. Then, the AWB mechanismcompares the average R value to the average G value and compares theaverage B value to the average G value to determine the R gainadjustment for the R channel, G gain adjustment for the G channel, and Bgain adjustment for B channel for white balancing.

The image sensor of an AWB system continuously captures imagessequentially. AWB performs white balancing on each captured image. Whitebalance settings are calculated for the current captured image andapplied on the subsequent captured image. The methodology continuouslywhite balances every incoming image to determine whether there is achange in the illuminant source and whether there is a need for gainadjustments.

Now turning to FIG. 1, which illustrates an image capture apparatus ofthe present invention. Color processing block 102 accepts input pixelsserially, such that several lines of an image may be access at one time.The input pixels are manipulated for later image display. Colorprocessing 102 also performs automatic white balancing.

Automatic white balance block 104 evaluates the input pixels todetermine the amount of gain adjustment necessary for each color channelto achieve white balance. Under different illuminant sources, thestrength of each RGB color will be dramatically different. When theilluminant source is daylight (i.e., the image is captured in daylight)the image will have a greater blue color component influence. Likewise,when the illuminant source is incandescent light there will be a greaterred color influence in the image, as oppose to the same image capturedunder daylight. Automatic white balance is performed to make sure awhite object in an image appears white under different illuminant typesby determining the gain adjustments, if needed, for the red channel,green channel, and blue channel.

White balancing is attained when the RGB values are equivalent. Gainblock 106 adjusts the red color value, if needed, in the red channel forwhite balancing. Similarly, gain block 108 adjusts the green colorvalue, if needed, in the green channel and gain block 110 adjusts theblue color value, if needed, in the blue channel.

Storage display 112 holds an image until the user wants to view theimage or displays the image in real time. For example, a camera system,in real time mode, may not store a whole image but instead seriallytransmits the pixels of the image to a computer or other displaysystems.

FIG. 2 illustrates one embodiment of a color processing system withautomatic white balance for one of the color channels. Gain block 202applies the associated gain adjustment to the color channel, if neededfor white balancing.

Gamma 204 controls the overall brightness as well as accurate colorreproduction of an image. If an image is not gamma corrected it can lookeither bleached out or too dark. The amount of gamma correction affectsthe brightness of an image and the ratios of red to green to blue. Gamma204 compensates for the non-linear relationship of pixel value andintensity of a display system before displaying the image.

Color processing module accepts input pixels serially, line by line. Aline of pixel input sequence is blue, green, blue, green, etc., and thenext line of pixel input sequence is green, red, green, red, etc. Thetotal number of pixels includes alternating lines of blue, green andgreen, red for the entire image with 50% green, 25% red, and 25% blue.Color interpolation 206 ascertains the two missing color values in eachpixel. There are several interpolation techniques, such as nearestneighbor, linear, cubic, and cubic spline. The output of colorinterpolation 206 is lines of RGB, RGB, RGB, etc.

Gamma correction and analog gain are for the display system. Therefore,if input data to AWB has been gamma corrected and gain adjusted theymust be reversed. Gamma function and analog gain may influence thelocation and profile of white areas. The white point location iscalibrated under standard gain setting and non-gamma transfer function.Anti-gamma 208 is used to cancel the effect of gamma correction.Anti-gain 210 is used to cancel the effect of the analog gains appliedin the R, G, and B channels.

Area selection block 212 calculates the Green/Red (G/R) ratio and theGreen/Blue (G/B) ratio of a pixel. For each pixel, the G/R and G/Bratios create a point to be plotted on a color space diagram. If thepoint falls within one of the predefined white areas in a color spacediagram then the pixel is a white pixel. For example, a white pixel thatfalls within the daylight predefined white area will have requisite G/Rand G/B ratios that are inside the predefined white area for daylight.

A pixel with R=100, G=105, and B=112 will result inG/R=(105/100)*128=134.5 and G/B=(105/112)*128=120. These ratios create apoint that falls within the predefined white area of CWF. Thus, theratios reveal that the pixel is a white pixel. In contrast, if R=225,G=10, and B=10 then G/R=(10/255)*128=5 and G/B=(10/255)*128=5, whichwould create a point outside of any predefined white area and the pixelcould not be a white pixel. If these RGB values are used in calculatingthe white balance setting, they will skew the true color of the otherpixels causing them to be too blue or too green.

When area selection block 212 finishes analyzing all pixels of an image.It then determines the illuminant source of the image by counting thenumber of white pixel points that are in each of the predefined whiteareas. The predefined white area with the highest number of white pixelpoints is indicative of the illuminant source of the image. For example,if the daylight predefined white area contains more white pixel pointsthan the CWF and A/U30 white areas then the illuminant source of thecaptured image is daylight. If the CWF white area has the highest numberof white pixel points then CWF is the illuminant source. Similarly, ifA/U30 has the highest number of white pixel points then the illuminantsource is incandescent light/general office light.

Accumulate for averaging block 214 stores all the white pixel RGB valuesuntil area selection block 212 analyzes all the pixels of an image.

Decide gain value block 216 calculates the average red value, averagegreen value, and average blue value of the white pixels of one, some, orall of the illuminant sources. The decide gain value block 216 uses thered, green, and blue averages to decide whether any color gainadjustment is required for white balancing.

In other embodiments, the color processing system applies the gamma andgain function outside of the AWB control loop. For example, gamma can beperformed in the Y channel or gain can be applied after colorinterpolation. If the gamma and gain functions are located in adifferent signal path then anti-gamma and anti-gain functions are notneeded in the system.

FIG. 3 illustrates a color space diagram with predefined white areas fordaylight 302, CWF 304, and A/U30 306. Although, FIG. 3 illustrates acolor space diagram with three predefined white areas, a color spacediagram of the present invention can have two or more predefined whiteareas. Further, FIG. 3 shows a combined predefined white area forilluminant types A (incandescent light) and U30 (general office light).A color space diagram can have separate predefined white area for A andU30, such that each illuminant type having its own predefined whitearea.

FIG. 4 depicts a color chart containing 24 blocks of different colors. Acolor chart is used for predefining white areas in a color space diagramfor different illuminant sources. A color chart is not limited to 24color blocks, but rather it can have any number of color blocksnecessary for defining white areas, so long as each of the blockscontains a known color, of which six of the blocks are the color white,gray 1, gray 2, gray 3, gray 4, and black. Gray 1 to gray 4 color blockcontains varying shades of gray.

FIG. 5 illustrates method 500 for predefining white areas in a colorspace diagram. Defining a white area for a target illuminant type in acolor space diagram requires analyzing a color chart under thatilluminant type. For example, to define a white area for daylightinvolves analyzing a color chart under daylight. Defining a white areain a color space diagram begins with step 502, calculating a G/R ratioand a G/B ratio for a color block under the target illuminant source(e.g., daylight, CWF, A, U30, etc.). Next, step 504 plots the G/R andG/B ratios of the color block on a color space diagram. Then, steps 502and 504 are repeated for all color blocks in the color chart. Step 508identifies the area defined by the G/R and G/B ratios of the white, gray1, gray 2, gray 3, and gray 4 color blocks in the color space diagram.This area is the predefined white area of the target illuminant source.Steps 502 to 508 are repeated for each illuminant source needed in acolor space diagram.

This methodology is advantageous as this technique eliminates anyinfluence from strong colors. Additionally, this technique can quicklydetect illuminant source change in an image and respond quickly to thechange by moving to a color gain setting that fits the illuminantsource. While under normal condition when the illuminant source isunchanged, determination of the color gain adjustment could be performedless quickly based on the average values of the white pixels in aselected white area. This robust technique supports rapid response toilluminant source change as well as provides stability in normaloperating condition.

While the preferred embodiment of the invention has been illustrated anddescribed, it will be appreciated that various changes can be madetherein without departing from the spirit and scope of the invention.

1. A method of automatic white balancing comprising: (a) determining anilluminant source by identifying a predefined white area of a colorspace diagram having a highest number of pixels; (b) calculating anaverage R value, an average G value, and an average Blue value of saidpixels; and (c) determining a gain adjustment based on said average Rvalue, said average G value, and said average B value.
 2. The method ofclaim 1, wherein said pixels are white pixels.
 3. The method of claim 2,further including the step of calculating a G/R ratio and a G/B ratio ofsaid pixels.
 4. The method of claim 3, wherein said G/R ratio and saidG/B ratio of said pixels are plotted on said color space diagram.
 5. Themethod of claim 4, wherein said R value, said G value, and said B valueare accumulated for said pixels.
 6. A method of identifying anilluminant source of a captured image for automatic white balancecomprising: (a) calculating a G/R ratio and a G/B ratio for a pixel ofsaid captured image; (b) plotting said G/R ratio and said G/B ratio in acolor space diagram; and (c) determining a predefined white area of saidcolor space diagram having a highest number of said pixels, which isindicative of said illuminant source of said captured image.
 7. Themethod of claim 7, wherein said pixels are white pixels.
 8. A method ofdetermining a gain adjustment for automatic white balance comprising:(a) calculating an average R value, an average G value, and an averageBlue value of a pixel of a captured image; and (b) determining a gainadjustment based on said average R value, said average G value, and saidaverage B value.
 9. The method of claim 8, wherein said pixel is aplurality of selected white pixels of a predefined white area.
 10. Amethod of automatic white balancing comprising: (a) calculating a G/Rratio and a G/B ratio for a pixel; (b) plotting said G/R ratio and saidG/B ratio in a color space diagram; (c) accumulating a R value, a Gvalue, and a Blue value for each said pixel that has said G/R ratio andsaid G/B ratio inside a predefined white area of said color spacediagram; (d) determining an illuminant source by identifying saidpredefined white area containing a highest number of said pixels; (e)calculating said R value average, said G value average, and said B valueaverage; and (f) determining a gain adjustment based on said R valueaverage, said G value average, and said B value average.
 11. A method ofpredefining a white area in a color space diagram for automatic whitebalance comprising: (a) calculating a G/R ratio and a G/B ratio for awhite color block; (b) repeating step (a) for each illuminant type; and(c) determining a white area for each said illuminant type based on saidG/R ratio and said G/B ratio for said white color block.
 12. The methodof claim 11, further including a plurality of color blocks of differentcolors.
 13. The method of claim 12, wherein said color blocks includinga plurality of gray color blocks of different shades.
 14. The method ofclaim 13, wherein steps (a) and (b) are repeated for each of said graycolor block.
 15. The method of claim 14, wherein said white area isdefined by said G/R ratio and said G/B ratio of said white color blockand said gray color blocks.
 16. A method of predefining a white area ina color space diagram for automatic white balancing comprising: (a)using a color chart having a plurality of color blocks including awhite, a gray 1, a gray 2, a gray 3, a gray 4, and a black color blockunder a target illuminant source; (b) calculating a G/R ratio and a G/Bratio for each said color block; (c) plotting said G/R ratio and saidG/B ratio of each said color block on said color space diagram; (d)defining said white area on said color space diagram for said targetilluminant source based upon said G/R ratio and said G/B ratio for saidwhite, said gray 1, said gray 2, said gray 3, and said gray 4 colorblocks; and (e) repeating steps (a) through (c) for each said targetilluminant source.
 17. An apparatus for automatic white balancecomprising: (a) an area selection module for determining a predefinedwhite area of a color space diagram for a pixel; (b) an accumulate foraveraging module for storing a R value, a G value, and a Blue value ofsaid pixel; and (c) a decide gain value module for determining a gainadjustment.
 18. The method of claim 17, wherein said area selectionmodule calculates a G/R ratio and a G/B ratio of said pixel.
 19. Themethod of claim 18, wherein said area selection module analyzes saidpredefined white area to identify said predefined white area having ahighest number of said pixel, which is indicative of said illuminantsource.
 20. The method of claim 19, wherein said decide gain valuemodule calculates an average R value, an average G value, and an averageB value of said pixel for said gain adjustment for a color channel.